Section: Application Domains

Augmented head

Visual information on a speaker, especially jaws and lips but also tongue position, noticeably improves speech intelligibility. Hence, having a realistic augmented head displaying both external and internal articulators could help language learning technology progress in giving the student a feedback on how to change articulation in order to achieve a correct pronunciation. The long term aim of the project is the acquisition of articulatory data and the design of a 3D+t articulatory model from various image modalities: external articulators are extracted from stereovision data, the tongue shape is acquired through ultrasound imaging, 3D images of all articulators can be obtained with MRI for sustained sounds, magnetic sensors are used to recover the tip of the tongue.